Severe Weather Data from NOAA 1950-2019

by: Kaegan Casey, Patrick Durkin, Mengchao Li, Yongbo Shu, Yiwei Yan

data: https://www1.ncdc.noaa.gov/pub/data/swdi/stormevents/csvfiles/

---

Navigation:

Introduction:

1. Import Libraries and read in data
2. Descriptive Statistics
3. Cleaning/Preparation

Visualizations:

4. Time Series Plots
5. Summary Plots

Conclusions:

6. T-tests
   • Overall
   • Individual Storms
7. Time Series Analysis
8. Predictive Modelling

Extras:

7. Creating our csv

---

back to navigation

Import libraries and read in data

library(ggplot2)

Registered S3 method overwritten by 'dplyr':
  method           from
  print.rowwise_df     

library(plyr)
library(dplyr)

Attaching package: ‘dplyr’

The following objects are masked from ‘package:plyr’:

    arrange, count, desc, failwith, id, mutate, rename, summarise, summarize

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union

library(quantmod)

Loading required package: xts
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Registered S3 method overwritten by 'xts':
  method     from
  as.zoo.xts zoo 

Attaching package: ‘xts’

The following objects are masked from ‘package:dplyr’:

    first, last

Loading required package: TTR
Registered S3 method overwritten by 'quantmod':
  method            from
  as.zoo.data.frame zoo 
Version 0.4-0 included new data defaults. See ?getSymbols.

library(scales)
library(rlist)

Registered S3 method overwritten by 'data.table':
  method           from
  print.data.table     

library(tidyverse)

── Attaching packages ──────────────────────────────────────────────────────────────── tidyverse 1.2.1 ──
✔ tibble  2.1.3     ✔ purrr   0.3.2
✔ tidyr   0.8.3     ✔ stringr 1.4.0
✔ readr   1.3.1     ✔ forcats 0.4.0
── Conflicts ─────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::arrange()    masks plyr::arrange()
✖ readr::col_factor() masks scales::col_factor()
✖ purrr::compact()    masks plyr::compact()
✖ dplyr::count()      masks plyr::count()
✖ purrr::discard()    masks scales::discard()
✖ dplyr::failwith()   masks plyr::failwith()
✖ dplyr::filter()     masks stats::filter()
✖ xts::first()        masks dplyr::first()
✖ dplyr::id()         masks plyr::id()
✖ dplyr::lag()        masks stats::lag()
✖ xts::last()         masks dplyr::last()
✖ dplyr::mutate()     masks plyr::mutate()
✖ dplyr::rename()     masks plyr::rename()
✖ dplyr::summarise()  masks plyr::summarise()
✖ dplyr::summarize()  masks plyr::summarize()

library(transformr)
library(gganimate)
library(xtable)

options(stringsAsFactors = FALSE, scipen=99999)

# weatherFull just needs to be in the same folder as this notebook .rmb file
full <- read.csv('weatherFull.csv')

---

back to navigation

Descriptive Statistics

head(full)

dim(full)

[1] 1572907      52

summary(full[,1:26])

       X           BEGIN_YEARMONTH    BEGIN_DAY       BEGIN_TIME   END_YEARMONTH       END_DAY     
 Min.   :      1   Min.   :195001   Min.   : 1.00   Min.   :   0   Min.   :195001   Min.   : 1.00  
 1st Qu.: 393228   1st Qu.:199905   1st Qu.: 7.00   1st Qu.: 900   1st Qu.:199905   1st Qu.: 8.00  
 Median : 786454   Median :200612   Median :15.00   Median :1515   Median :200612   Median :16.00  
 Mean   : 786454   Mean   :200422   Mean   :15.02   Mean   :1324   Mean   :200422   Mean   :16.35  
 3rd Qu.:1179680   3rd Qu.:201212   3rd Qu.:23.00   3rd Qu.:1820   3rd Qu.:201212   3rd Qu.:24.00  
 Max.   :1572907   Max.   :201906   Max.   :31.00   Max.   :2359   Max.   :201906   Max.   :31.00  
                                                                                                   
    END_TIME      EPISODE_ID           EVENT_ID            STATE             STATE_FIPS        YEAR     
 Min.   :   0   Min.   :        1   Min.   :        3   Length:1572907     Min.   : 1.0   Min.   :1950  
 1st Qu.:1200   1st Qu.:    59043   1st Qu.:   415503   Class :character   1st Qu.:19.0   1st Qu.:1999  
 Median :1613   Median :   117466   Median :   831695   Mode  :character   Median :30.0   Median :2006  
 Mean   :1483   Mean   :   637232   Mean   :  3603266                      Mean   :31.7   Mean   :2004  
 3rd Qu.:1909   3rd Qu.:  1163492   3rd Qu.:  5562408                      3rd Qu.:45.0   3rd Qu.:2012  
 Max.   :2359   Max.   :990000001   Max.   :990000003                      Max.   :99.0   Max.   :2019  
                NA's   :232239                                             NA's   :1                    
  MONTH_NAME         EVENT_TYPE          CZ_TYPE             CZ_FIPS        CZ_NAME         
 Length:1572907     Length:1572907     Length:1572907     Min.   :  0.0   Length:1572907    
 Class :character   Class :character   Class :character   1st Qu.: 25.0   Class :character  
 Mode  :character   Mode  :character   Mode  :character   Median : 65.0   Mode  :character  
                                                          Mean   : 94.2                     
                                                          3rd Qu.:117.0                     
                                                          Max.   :876.0                     
                                                                                            
     WFO            BEGIN_DATE_TIME    CZ_TIMEZONE        END_DATE_TIME      INJURIES_DIRECT    
 Length:1572907     Length:1572907     Length:1572907     Length:1572907     Min.   :   0.0000  
 Class :character   Class :character   Class :character   Class :character   1st Qu.:   0.0000  
 Mode  :character   Mode  :character   Mode  :character   Mode  :character   Median :   0.0000  
                                                                             Mean   :   0.0967  
                                                                             3rd Qu.:   0.0000  
                                                                             Max.   :1700.0000  
                                                                                                
 INJURIES_INDIRECT   DEATHS_DIRECT      DEATHS_INDIRECT    DAMAGE_PROPERTY   
 Min.   :   0.0000   Min.   :  0.0000   Min.   : 0.00000   Length:1572907    
 1st Qu.:   0.0000   1st Qu.:  0.0000   1st Qu.: 0.00000   Class :character  
 Median :   0.0000   Median :  0.0000   Median : 0.00000   Mode  :character  
 Mean   :   0.0084   Mean   :  0.0116   Mean   : 0.00161                     
 3rd Qu.:   0.0000   3rd Qu.:  0.0000   3rd Qu.: 0.00000                     
 Max.   :2400.0000   Max.   :638.0000   Max.   :20.00000                     
                                                                             

summary(full[,27:52])

 DAMAGE_CROPS          SOURCE            MAGNITUDE       MAGNITUDE_TYPE     FLOOD_CAUSE       
 Length:1572907     Length:1572907     Min.   :    0.0   Length:1572907     Length:1572907    
 Class :character   Class :character   1st Qu.:    0.9   Class :character   Class :character  
 Mode  :character   Mode  :character   Median :    1.8   Mode  :character   Mode  :character  
                                       Mean   :   25.5                                        
                                       3rd Qu.:   52.0                                        
                                       Max.   :22000.0                                        
                                       NA's   :653424                                         
    CATEGORY       TOR_F_SCALE          TOR_LENGTH        TOR_WIDTH       TOR_OTHER_WFO     
 Min.   :1.0       Length:1572907     Min.   :   0.0    Min.   :   0.0    Length:1572907    
 1st Qu.:1.0       Class :character   1st Qu.:   0.0    1st Qu.:   0.0    Class :character  
 Median :1.0       Mode  :character   Median :   0.0    Median :   0.0    Mode  :character  
 Mean   :1.4                          Mean   :   0.9    Mean   :  32.3                      
 3rd Qu.:1.0                          3rd Qu.:   0.0    3rd Qu.:  10.0                      
 Max.   :5.0                          Max.   :2315.0    Max.   :5280.0                      
 NA's   :1572581                      NA's   :1307840   NA's   :1307840                     
 TOR_OTHER_CZ_STATE TOR_OTHER_CZ_FIPS TOR_OTHER_CZ_NAME   BEGIN_RANGE     BEGIN_AZIMUTH     
 Length:1572907     Min.   :  1.0     Length:1572907     Min.   :   0.0   Length:1572907    
 Class :character   1st Qu.: 47.0     Class :character   1st Qu.:   0.0   Class :character  
 Mode  :character   Median : 93.0     Mode  :character   Median :   1.0   Mode  :character  
                    Mean   :104.6                        Mean   :   2.4                     
                    3rd Qu.:139.0                        3rd Qu.:   3.0                     
                    Max.   :810.0                        Max.   :3749.0                     
                    NA's   :1570665                      NA's   :716340                     
 BEGIN_LOCATION       END_RANGE      END_AZIMUTH        END_LOCATION         BEGIN_LAT     
 Length:1572907     Min.   :  0.0    Length:1572907     Length:1572907     Min.   :-14.4   
 Class :character   1st Qu.:  0.0    Class :character   Class :character   1st Qu.: 34.4   
 Mode  :character   Median :  1.0    Mode  :character   Mode  :character   Median : 38.0   
                    Mean   :  2.3                                          Mean   : 37.8   
                    3rd Qu.:  3.0                                          3rd Qu.: 41.3   
                    Max.   :925.0                                          Max.   : 70.5   
                    NA's   :716717                                         NA's   :599776  
   BEGIN_LON         END_LAT          END_LON       EPISODE_NARRATIVE  EVENT_NARRATIVE   
 Min.   :-171.0   Min.   :-14.5    Min.   :-815.1   Length:1572907     Length:1572907    
 1st Qu.: -97.5   1st Qu.: 34.6    1st Qu.: -97.5   Class :character   Class :character  
 Median : -91.4   Median : 38.0    Median : -91.0   Mode  :character   Mode  :character  
 Mean   : -91.0   Mean   : 37.9    Mean   : -92.6                                        
 3rd Qu.: -83.5   3rd Qu.: 41.3    3rd Qu.: -83.2                                        
 Max.   : 171.4   Max.   : 70.4    Max.   : 171.4                                        
 NA's   :599776   NA's   :777266   NA's   :777265                                        
 DATA_SOURCE       
 Length:1572907    
 Class :character  
 Mode  :character  
                   
                   
                   
                   

length(unique(full$EVENT_TYPE))

[1] 74

---

back to navigation

Cleaning/Preparation

seperate BEGIN_YEARMONTH using a “-” and create new variable YMD that contains the year month and day seperated by a “-”

full$BEGIN_YEARMONTH <- gsub("(\\d{4})(\\d{2})", "\\1-\\2", full$BEGIN_YEARMONTH)
full$YMD <- paste(full$BEGIN_YEARMONTH, full$BEGIN_DAY, sep="-")
head(full)

Clean up DAMAGE_PROPERTY column so it can be used numerically and save as new variable

xin1 <- xin2 <- xin3 <- rep(NA,times=length(full$DAMAGE_PROPERTY))


for (i in 1:length(full$DAMAGE_PROPERTY)){
  if (grepl("K",full$DAMAGE_PROPERTY[i])==T){
    xin1[i] <- sub("K","",full$DAMAGE_PROPERTY[i])
  }
  if (grepl("M",full$DAMAGE_PROPERTY[i])==T){
    xin2[i] <- sub("M","",full$DAMAGE_PROPERTY[i])
  }
}
xin1 <- as.numeric(xin1)
xin2 <- as.numeric(xin2)
xin1 <- xin1 * 1000
xin2 <- xin2 * 1000000
xin1[is.na(xin1)] <- 0
xin2[is.na(xin2)] <- 0

xin3 <- xin1 + xin2
full$DAMAGE_PROPERTY_CLEAN <- xin3
head(full)

Adjust DAMAGE_PROPERTY_CLEAN column for inflation

#   using quantmod package to access consumer price index data
getSymbols("CPIAUCSL", src='FRED')

‘getSymbols’ currently uses auto.assign=TRUE by default, but will
use auto.assign=FALSE in 0.5-0. You will still be able to use
‘loadSymbols’ to automatically load data. getOption("getSymbols.env")
and getOption("getSymbols.auto.assign") will still be checked for
alternate defaults.

This message is shown once per session and may be disabled by setting 
options("getSymbols.warning4.0"=FALSE). See ?getSymbols for details.

[1] "CPIAUCSL"

cpi <- CPIAUCSL

cpiMonths <- substr(index(cpi), 1, 7)

cpidf <- data.frame("months" = cpiMonths, "cpi" = cpi)
names(cpidf) <- c("months", "CPI")

full <- merge(full, cpidf, by.x= "BEGIN_YEARMONTH", by.y= "months")

# our dataset has the most recent storm recorded being in july when the 
# cpi was 255 so we are grabbing the cpi as of oct 1st (most recent for todays dollar)
cpitoday <- tail(cpi, 1)

full$multFactor <- as.vector(cpitoday) / full$CPI 

full$ADJ_DAMAGE_PROPERTY <- full$DAMAGE_PROPERTY_CLEAN * full$multFactor
head(full)

---

back to navigation

Time Series Plots

Direct Deaths by Month

one outlier that was over 1000 removed

d_by_m <- aggregate( DEATHS_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=full )
names(d_by_m) <- c("BEGIN_YEARMONTH", "DEATHS_DIRECT")

windsor <- subset(d_by_m, DEATHS_DIRECT < 750)

n <- 20
numrow <- nrow(d_by_m)/n
seq1 <- seq(from=numrow, to=numrow*n-1, by=numrow)

ggplot(windsor, aes(BEGIN_YEARMONTH, DEATHS_DIRECT))+
  geom_point(size=.8)+
  scale_x_discrete(name="Year-Month", breaks=windsor$BEGIN_YEARMONTH[seq1])+
  ggtitle("Directly Related Deaths by Month over 70 years", subtitle = "*one outlier removed")+
  theme(plot.title = element_text(hjust = 0.5), plot.subtitle = element_text(hjust = 0.5))+
  ylab("Directly Related Deaths")

Directly related injuries per month

d_by_m <- aggregate( INJURIES_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=full )
names(d_by_m) <- c("BEGIN_YEARMONTH", "INJURIES_DIRECT")

#   create a sequence for plotting a certain number (n) of discrete x variable dates, used to index year month in scale_x_discrete()
n <- 20
numrow <- nrow(d_by_m)/n
seq1 <- seq(from=numrow, to=numrow*n-1, by=numrow)

#   PLOT!
ggplot(d_by_m, aes(BEGIN_YEARMONTH, INJURIES_DIRECT))+
  geom_point(size=.8)+
  scale_x_discrete(name="Year-Month", breaks=d_by_m$BEGIN_YEARMONTH[seq1])+
  ggtitle("Directly Related Injuries by Month over 70 years")+
  theme(plot.title = element_text(hjust = 0.5))+
  ylab("Directly Related Injuries")

Damaged property per month

#   aggregate deaths by year and month
d_by_m <- aggregate( ADJ_DAMAGE_PROPERTY ~ BEGIN_YEARMONTH, FUN=sum, data=full )
names(d_by_m) <- c("BEGIN_YEARMONTH", "ADJ_DAMAGE_PROPERTY")

#   create a sequence for plotting a certain number (n) of discrete x variable dates, used to index year month in scale_x_discrete()
n <- 20
numrow <- nrow(d_by_m)/n
seq1 <- seq(from=numrow, to=numrow*n-1, by=numrow)

#   PLOT!
ggplot(d_by_m, aes(BEGIN_YEARMONTH, ADJ_DAMAGE_PROPERTY))+
  geom_point(size=.8)+
  scale_x_discrete(name="Year-Month", breaks=d_by_m$BEGIN_YEARMONTH[seq1])+
  ggtitle("Property Damage by Month over 70 years")+
  theme(plot.title = element_text(hjust = 0.5))+
  ylab("Adjusted for Inflation Damage to Property ($)")+
  scale_y_continuous(label=comma)

Most dangerous storm deaths by day

here we can very clearly see that they started recording certain types of storms at different points in time

#   select relevant columns
cols <- c("YMD", "DEATHS_DIRECT", "EVENT_TYPE", "YEAR")
work <- full[,cols]

#   get most dangerous types of storms having DEATHS_DIRECT > third quartile
e_by_d <- aggregate(DEATHS_DIRECT ~ EVENT_TYPE, FUN = sum, data=full)
sub <- subset(e_by_d, DEATHS_DIRECT > 50) 

#   filter by most dangerous storm found above
#   DEATHS_DIRECT < 100 removes one outlier that affects all of the graphs in the facet wrap so I removed it
new <- subset(work, EVENT_TYPE %in% sub$EVENT_TYPE & DEATHS_DIRECT < 100)   

ggplot(new, aes(YMD, DEATHS_DIRECT))+
  geom_point(size=.5)+
  facet_wrap(vars(new$EVENT_TYPE))+
  ggtitle("Deaths Over Time for Most Dangerous Types of Events")+
  xlab("Year-Month-Day")+
  theme_minimal()+
  theme(plot.title = element_text(hjust = 0.5), axis.text.x=element_blank())

Barplots of directly related deaths by storm type for every year in the data set

saved as two seperate large pdfs for viewability

#   aggregate full dataset by year and event type finding the sum of directly related deaths
agg <- aggregate(DEATHS_DIRECT ~ YEAR + EVENT_TYPE, FUN=sum, data=full)
agg <- agg[order(agg$YEAR),]

#   create year sequences
seq1 <- 1950:1995
seq2 <- 1996:2019

firstSet <- agg[agg$YEAR %in% seq1,]
secondSet <- agg[agg$YEAR %in% seq2,]

#   Since a bunch of new EVENT_TYPES were added around 1955 there were a lot of extra ones that have 0 direct deaths associated with them so they were filtered out for viewing purposes.
firstSet <- subset(firstSet, DEATHS_DIRECT > 0)

#   this is based off of the 3rd qurtile of DEATHS_DIRECT and is meant to narrow it down to more dangerous types of events because there are so many types of events it is for viewing purposes.
secondSet <- subset(secondSet, DEATHS_DIRECT > summary(secondSet$DEATHS_DIRECT)[5])


#   PLOT!!!!
#   1950-1995
ggplot(firstSet, aes(EVENT_TYPE, DEATHS_DIRECT))+
  geom_bar(stat = 'identity')+
  coord_flip()+
  facet_wrap(vars(firstSet$YEAR))+
  ggtitle("Directly Related Deaths by Event Type", subtitle = "1950-1995")+
  theme(plot.title = element_text(hjust = 0.5), plot.subtitle = element_text(hjust = 0.5))+
  xlab("Directly Related Deaths")+
  ylab("Event Type")

ggsave('deaths_by_EVENT_TYPE_1950-1995.pdf')

Saving 30 x 16 in image

#   1996-2019
 ggplot(secondSet, aes(EVENT_TYPE, DEATHS_DIRECT))+
  geom_bar(stat = 'identity')+
  coord_flip()+
  facet_wrap(vars(secondSet$YEAR))+
  ggtitle("Directly Related Deaths by Event Type", subtitle = "1996-2019")+
  theme(plot.title = element_text(hjust = 0.5), plot.subtitle = element_text(hjust = 0.5))+
  xlab("Directly Related Deaths")+
  ylab("Event Type")

ggsave('deaths_by_EVENT_TYPE_1996-2019.pdf')

Saving 30 x 16 in image

Frequency of storm events for every year in data set

saved as two seperate large pdfs for viewability

#   used to get the count of each event type by year, from dplyr package
byYear <- full %>% count(YEAR, EVENT_TYPE)

#   this is to limit the graph to storms that are most frequent using the first quartile
byYear <- subset(byYear, n > summary(byYear$n)[2])  

seq1 <- 1950:1995
seq2 <- 1996:2019

firstSet <- byYear[byYear$YEAR %in% seq1,]
secondSet <- byYear[byYear$YEAR %in% seq2,]

#   the second set contains so many types of events it had to be limited to storms that are most frequent using the first quartile
secondSet <- subset(secondSet, n > summary(secondSet$n)[2])

#   PLOT!!!!
#   1953-1995
ggplot(firstSet, aes(EVENT_TYPE, n))+
  geom_bar(stat='Identity')+
  coord_flip()+
  facet_wrap(vars(firstSet$YEAR))+
  ggtitle("Frequency of Events by Year", subtitle = "1953-1995")+
  theme(plot.title = element_text(hjust = 0.5), plot.subtitle = element_text(hjust = 0.5))+
  ylab("Frequency")+
  xlab("Event Type")

ggsave("event_frequency_1953-1995.pdf")

Saving 30 x 16 in image

#   1996-2019
ggplot(secondSet, aes(EVENT_TYPE, n))+
  geom_bar(stat='Identity')+
  coord_flip()+
  facet_wrap(vars(secondSet$YEAR))+
  ggtitle("Frequency of Events by Year", subtitle = "1996-2019")+
  theme(plot.title = element_text(hjust = 0.5), plot.subtitle = element_text(hjust = 0.5))+
  ylab("Frequency")+
  xlab("Event Type")

ggsave("event_frequency_1996-2019.pdf")

Saving 30 x 16 in image

---

back to navigation

Summary plots

Most dangerous types of events in terms of directly related deaths

There are so many types of storms this data had to be subset to show the top most dangerous clearly

e_by_d <- aggregate(DEATHS_DIRECT ~ EVENT_TYPE, FUN = sum, data=full)
sub <- subset(e_by_d, DEATHS_DIRECT > 200)  # arbitrary cutoff point

ggplot(sub, aes(reorder(EVENT_TYPE, DEATHS_DIRECT), DEATHS_DIRECT))+
  geom_bar(stat = 'identity')+
  ggtitle("Most Dangerous Events in Terms of Deaths")+
  theme(plot.title = element_text(hjust = 0.5))+
  ylab("Directly Related Deaths")+
  xlab("Event Types")+
  scale_y_continuous(label=comma)+
  coord_flip()

e_by_d <- aggregate(INJURIES_DIRECT ~ EVENT_TYPE, FUN = sum, data=full)
sub <- subset(e_by_d, INJURIES_DIRECT > mean(e_by_d$INJURIES_DIRECT))  

ggplot(sub, aes(reorder(EVENT_TYPE, INJURIES_DIRECT), INJURIES_DIRECT))+
  geom_bar(stat = 'identity')+
  ggtitle("Most Dangerous Events in Terms of Injuries")+
  theme(plot.title = element_text(hjust = 0.5), plot.margin = margin(10, 15, 10, 10))+
  ylab("Directly Related Injuries")+
  xlab("Event Types")+
  scale_y_continuous(label=comma)+
  coord_flip()

Most Costly Events in terms of property damage

subset based on the mean for viewability

e_by_d <- aggregate(ADJ_DAMAGE_PROPERTY ~ EVENT_TYPE, FUN = sum, data=full)
sub <- subset(e_by_d, ADJ_DAMAGE_PROPERTY > mean(e_by_d$ADJ_DAMAGE_PROPERTY))  

ggplot(sub, aes(reorder(EVENT_TYPE, ADJ_DAMAGE_PROPERTY), ADJ_DAMAGE_PROPERTY))+
  geom_bar(stat = 'identity')+
  ggtitle("Most Costly Events in terms of Property Damage")+
  theme(plot.title = element_text(hjust = 0.5))+
  ylab("Property Damage Adjusted for Inflation ($)")+
  xlab("Event Types")+
  scale_y_continuous(label=comma)+
  coord_flip()

Storm Events based on frequency

#   obtain counts and subset out infrequent events using mean, again dplyr package
event <- full %>% count(EVENT_TYPE)
event <- subset(event, n >= mean(event$n)) 

#   PLOT!!!!
ggplot(event, aes(reorder(EVENT_TYPE, n), n))+
  geom_bar(stat='Identity')+
  ggtitle("Frequency of Events for the Past 70 Years")+
  theme(plot.title = element_text(hjust = 0.5))+
  ylab("Frequency")+
  xlab("Event Types")+
  coord_flip()+
  scale_y_continuous(labels = scales::comma)   # forces ggplot to stop using scientific notation for y-axis

Pie chart of directly related deaths by state

tib_data <- as_tibble(full)

# clean state tib_data
newdata <- subset(tib_data %>% select (STATE, DEATHS_DIRECT), STATE %in% toupper(state.name))

# rank state in terms of death
dang_state <- aggregate(DEATHS_DIRECT~STATE, data=newdata, sum)
dang_state$dang_rank <- rank(-dang_state$DEATHS_DIRECT, ties.method='min',)

dang_state <- dang_state[order(dang_state$dang_rank),]

# subset states with less than 100 deaths
above100 <- subset(dang_state, dang_state$DEATHS_DIRECT > 100)

# rstudio pie chart
pie(above100$DEATHS_DIRECT, labels=dang_state$STATE, cex=0.6, main ='Total Amount of Direct Death per State, 1950 - 2019')

# ggplot pie chart
piepie <- ggplot(dang_state, aes(x="", y=dang_state$DEATHS_DIRECT,fill=dang_state$DEATHS_DIRECT))

above100$prob <- above100$DEATHS_DIRECT / sum(above100$DEATHS_DIRECT) * 100

ggplot(above100, aes(x="", y=prob, fill=STATE)) +
  geom_bar(stat="identity", width=1) +
  coord_polar("y", start=0)

---

back to navigation

t.tests between 1996-2008 and 2009-2019 aggregated by Year-Month

NOAA only collected data on three types of severe storms until 1996 so in order to compare past data to more recent data we had to choose a date range that contained the same types of storms

Seperate data into two sets of years that contain similar types of events

yrs1 <- 1996:2007
yrs2 <- 2008:2019

early <- full[full$YEAR %in% yrs1,]
late <- full[full$YEAR %in% yrs2,]

d_early <- aggregate( DEATHS_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=early )
d_late <- aggregate( DEATHS_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=late )

i_early <- aggregate( INJURIES_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=early )
i_late <- aggregate( INJURIES_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=late )

p_early <- aggregate( ADJ_DAMAGE_PROPERTY ~ BEGIN_YEARMONTH, FUN=sum, data=early )
p_late <- aggregate( ADJ_DAMAGE_PROPERTY ~ BEGIN_YEARMONTH, FUN=sum, data=late )


# check that there is a similar amount in each subset of data, they all have the same # of rows
nrow(d_early)

[1] 144

nrow(d_late)

[1] 138

t.test of directly related deaths aggregated by year-month

H0 = there is no difference in directly related deaths between the two date ranges,

HA = there is a significant difference in directly related deaths between the two date ranges

hist(d_early$DEATHS_DIRECT)

hist(d_late$DEATHS_DIRECT)

t.test(d_early$DEATHS_DIRECT, d_late$DEATHS_DIRECT)

    Welch Two Sample t-test

data:  d_early$DEATHS_DIRECT and d_late$DEATHS_DIRECT
t = 0.86389, df = 207.67, p-value = 0.3886
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -9.739393 24.932630
sample estimates:
mean of x mean of y 
 52.44444  44.84783 

t.test of directly related deaths aggregated by year-month, outliers removed.

doesn’t change the conclusion just makes it less significant as one would expect.

thresh <- 100

d_earlyOut <- subset(early, DEATHS_DIRECT < thresh)
d_lateOut<- subset(late, DEATHS_DIRECT < thresh)

d_early <- aggregate( DEATHS_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=d_earlyOut )
d_late <- aggregate( DEATHS_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=d_lateOut )

summary(d_early$DEATHS_DIRECT)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   2.00   22.00   35.50   47.04   53.25  477.00 

summary(d_late$DEATHS_DIRECT)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
    5.0    20.0    33.5    43.7    53.0   413.0 

hist(d_early$DEATHS_DIRECT)

hist(d_late$DEATHS_DIRECT)

t.test(d_early$DEATHS_DIRECT, d_late$DEATHS_DIRECT)

    Welch Two Sample t-test

data:  d_early$DEATHS_DIRECT and d_late$DEATHS_DIRECT
t = 0.6014, df = 277.34, p-value = 0.5481
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -7.590042 14.267579
sample estimates:
mean of x mean of y 
 47.04167  43.70290 

t.test of directly related injuries aggregated by year-month

H0 = there is no difference in directly related injuries between the two date ranges,

HA = there is a significant difference in directly related injuries between the two date ranges

hist(i_early$INJURIES_DIRECT)

hist(i_late$INJURIES_DIRECT)

t.test(i_early$INJURIES_DIRECT, i_late$INJURIES_DIRECT)

    Welch Two Sample t-test

data:  i_early$INJURIES_DIRECT and i_late$INJURIES_DIRECT
t = 1.6673, df = 255.32, p-value = 0.09669
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -17.65705 212.59847
sample estimates:
mean of x mean of y 
 318.1736  220.7029 

t.test of directly related injuries, outliers removed

thresh <- 600

i_earlyOut <- subset(early, INJURIES_DIRECT < thresh)
i_lateOut<- subset(late, INJURIES_DIRECT < thresh)

i_early <- aggregate( INJURIES_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=i_earlyOut )
i_late <- aggregate( INJURIES_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=i_lateOut )

summary(i_early$INJURIES_DIRECT)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   14.0   108.5   209.5   303.6   354.0  4782.0 

summary(i_late$INJURIES_DIRECT)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   6.00   43.25   98.00  201.50  233.75 2178.00 

hist(i_early$INJURIES_DIRECT)

hist(i_late$INJURIES_DIRECT)

t.test(i_early$INJURIES_DIRECT, i_late$INJURIES_DIRECT)

    Welch Two Sample t-test

data:  i_early$INJURIES_DIRECT and i_late$INJURIES_DIRECT
t = 2.2046, df = 236.69, p-value = 0.02845
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  10.86082 193.31974
sample estimates:
mean of x mean of y 
 303.5903  201.5000 

t.test of damaged property aggregated by year-month

H0 = there is no difference in damaged property between the two date ranges,

HA = there is a significant difference in damaged property between the two date ranges

hist(p_early$ADJ_DAMAGE_PROPERTY)

hist(p_late$ADJ_DAMAGE_PROPERTY)

t.test(p_early$ADJ_DAMAGE_PROPERTY, p_late$ADJ_DAMAGE_PROPERTY)

    Welch Two Sample t-test

data:  p_early$ADJ_DAMAGE_PROPERTY and p_late$ADJ_DAMAGE_PROPERTY
t = -0.56822, df = 230.01, p-value = 0.5704
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -553018314  305446901
sample estimates:
mean of x mean of y 
849854710 973640417 

t.test of damaged property, outliers removed

thresh <- 1000000000

p_earlyOut <- subset(early, ADJ_DAMAGE_PROPERTY < thresh)
p_lateOut<- subset(late, ADJ_DAMAGE_PROPERTY < thresh)

p_early <- aggregate( ADJ_DAMAGE_PROPERTY ~ BEGIN_YEARMONTH, FUN=sum, data=p_earlyOut )
p_late <- aggregate( ADJ_DAMAGE_PROPERTY ~ BEGIN_YEARMONTH, FUN=sum, data=p_lateOut )

summary(p_early$ADJ_DAMAGE_PROPERTY)

       Min.     1st Qu.      Median        Mean     3rd Qu.        Max. 
   18506625   235335400   495839318   826047326   925555247 10794085770 

summary(p_late$ADJ_DAMAGE_PROPERTY)

       Min.     1st Qu.      Median        Mean     3rd Qu.        Max. 
   10750516   137834258   358143973   951391178   852358770 21480032375 

hist(p_early$ADJ_DAMAGE_PROPERTY)

hist(p_late$ADJ_DAMAGE_PROPERTY)

t.test(p_early$ADJ_DAMAGE_PROPERTY, p_late$ADJ_DAMAGE_PROPERTY)

    Welch Two Sample t-test

data:  p_early$ADJ_DAMAGE_PROPERTY and p_late$ADJ_DAMAGE_PROPERTY
t = -0.5933, df = 214.52, p-value = 0.5536
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -541765914  291078209
sample estimates:
mean of x mean of y 
826047326 951391178 

---

back to navigation

t.tests of individual storms comparing directly related deaths, injuries, and property damage

H0 = there is no difference in directly related deaths, injuries, or property damage between the two date ranges

HA = there is a significant difference in directly related deaths, injuries, or property damage between the two date ranges

write a function to t-test a list of different storms removing outliers and aggregating by year-month

test_storms <- function(events){
  
  return_list <- rep(NA, 1)
  return_count <- 1
  
  d_thresh <- 100
  i_thresh <- 600
  p_thresh <- 1000000000
  
  earlyOut <- subset(early, ADJ_DAMAGE_PROPERTY < p_thresh & INJURIES_DIRECT < i_thresh & DEATHS_DIRECT < d_thresh)
  lateOut<- subset(late, ADJ_DAMAGE_PROPERTY < p_thresh & INJURIES_DIRECT < i_thresh & DEATHS_DIRECT < d_thresh)
  
  for (event in events){
    eventEarly <- subset(earlyOut, EVENT_TYPE == event)
    eventLate <- subset(lateOut, EVENT_TYPE == event)
    
    d_eventEarly <- aggregate( DEATHS_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=eventEarly )
    d_eventLate <- aggregate( DEATHS_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=eventLate )
    
    i_eventEarly <- aggregate( INJURIES_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=eventEarly )
    i_eventLate <- aggregate( INJURIES_DIRECT ~ BEGIN_YEARMONTH, FUN=sum, data=eventLate )
    
    p_eventEarly <- aggregate( ADJ_DAMAGE_PROPERTY ~ BEGIN_YEARMONTH, FUN=sum, data=eventEarly )
    p_eventLate <- aggregate( ADJ_DAMAGE_PROPERTY ~ BEGIN_YEARMONTH, FUN=sum, data=eventLate )
    
    # cannot reject null
    d_test <- t.test(d_eventEarly$DEATHS_DIRECT, d_eventLate$DEATHS_DIRECT)
    
    # cannot reject null
    i_test <- t.test(i_eventEarly$INJURIES_DIRECT, i_eventLate$INJURIES_DIRECT)
    
    # cannot reject null
    p_test <- t.test(p_eventEarly$ADJ_DAMAGE_PROPERTY, p_eventLate$ADJ_DAMAGE_PROPERTY)
    
    test_list <- list("event" = event, "deaths" = d_test, "injuries" = i_test, "prop_damage" = p_test)
    return_list <- list.append(return_list, test_list)
  }
  return_list <- return_list[!is.na(return_list)]
  return(return_list)
}

create list and use function

events <- c("Hurricane (Typhoon)", "Tornado", "Heat", "Flash Flood", "Flood", 
            "Thunderstorm Wind", "Hail", "High Wind", "Wildfire", "Lightning", "Rip Current")
test_storms(events)

$event
[1] "Hurricane (Typhoon)"

$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = 1.277, df = 42.471, p-value = 0.2085
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -3.413073 15.185683
sample estimates:
mean of x mean of y 
6.4418605 0.5555556 


$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 1.3553, df = 48.245, p-value = 0.1816
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -5.397831 27.733748
sample estimates:
mean of x mean of y 
14.279070  3.111111 


$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = 1.0191, df = 55.571, p-value = 0.3126
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -330496245 1014695271
sample estimates:
mean of x mean of y 
609738096 267638584 


[[5]]
[[5]]$event
[1] "Tornado"

[[5]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = -0.37366, df = 175.53, p-value = 0.7091
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -5.712206  3.893517
sample estimates:
mean of x mean of y 
 5.097902  6.007246 


[[5]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 0.36112, df = 239.51, p-value = 0.7183
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -34.2825  49.6732
sample estimates:
mean of x mean of y 
 81.60839  73.91304 


[[5]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = 0.12992, df = 248.58, p-value = 0.8967
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -65345667  74575637
sample estimates:
mean of x mean of y 
110780089 106165104 



[[6]]
[[6]]$event
[1] "Heat"

[[6]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = 1.8563, df = 124.06, p-value = 0.06578
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.5658258 17.6478964
sample estimates:
mean of x mean of y 
15.787611  7.246575 


[[6]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 0.67914, df = 175.66, p-value = 0.4979
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -27.52565  56.40940
sample estimates:
mean of x mean of y 
 60.31858  45.87671 


[[6]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = -0.6759, df = 126.58, p-value = 0.5003
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -285209.6  139982.8
sample estimates:
mean of x mean of y 
 82425.24 155038.60 



[[7]]
[[7]]$event
[1] "Flash Flood"

[[7]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = -0.44784, df = 253.6, p-value = 0.6547
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -2.17306  1.36785
sample estimates:
mean of x mean of y 
 5.111888  5.514493 


[[7]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 1.3673, df = 143.01, p-value = 0.1737
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -12.5941  69.1127
sample estimates:
mean of x mean of y 
31.643357  3.384058 


[[7]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = -0.97729, df = 142.32, p-value = 0.3301
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -382043311  129261216
sample estimates:
mean of x mean of y 
115152263 241543310 



[[8]]
[[8]]$event
[1] "Flood"

[[8]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = -2.1665, df = 263.37, p-value = 0.03117
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -1.54113369 -0.07360061
sample estimates:
mean of x mean of y 
 1.569444  2.376812 


[[8]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 1.0787, df = 144.47, p-value = 0.2825
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -11.11674  37.82749
sample estimates:
mean of x mean of y 
14.993056  1.637681 


[[8]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = -1.3821, df = 209.88, p-value = 0.1684
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -109984729   19325393
sample estimates:
mean of x mean of y 
 83194105 128523773 



[[9]]
[[9]]$event
[1] "Thunderstorm Wind"

[[9]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = -1.6684, df = 242.41, p-value = 0.09653
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -1.5018257  0.1244102
sample estimates:
mean of x mean of y 
 1.840278  2.528986 


[[9]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 2.4659, df = 247.7, p-value = 0.01435
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  1.884546 16.842508
sample estimates:
mean of x mean of y 
 27.47222  18.10870 


[[9]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = 0.22651, df = 240.31, p-value = 0.821
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -28333369  35695952
sample estimates:
mean of x mean of y 
 47547664  43866373 



[[10]]
[[10]]$event
[1] "Hail"

[[10]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = 1.398, df = 187.57, p-value = 0.1638
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.01127332  0.06611922
sample estimates:
 mean of x  mean of y 
0.03472222 0.00729927 


[[10]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 1.5914, df = 203.64, p-value = 0.1131
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.4471504  4.1900539
sample estimates:
mean of x mean of y 
 3.944444  2.072993 


[[10]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = -0.14622, df = 268.97, p-value = 0.8839
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -56047706  48298386
sample estimates:
mean of x mean of y 
 87044456  90919116 



[[11]]
[[11]]$event
[1] "High Wind"

[[11]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = 1.7502, df = 233.79, p-value = 0.08139
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.07463717  1.26243911
sample estimates:
mean of x mean of y 
1.5069444 0.9130435 


[[11]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 3.9239, df = 268.28, p-value = 0.0001108
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 2.30633 6.95152
sample estimates:
mean of x mean of y 
 7.534722  2.905797 


[[11]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = -0.29487, df = 245.2, p-value = 0.7683
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -62232001  46025257
sample estimates:
mean of x mean of y 
 32896616  40999988 



[[12]]
[[12]]$event
[1] "Wildfire"

[[12]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = -1.0103, df = 154.84, p-value = 0.3139
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -2.3586744  0.7624178
sample estimates:
mean of x mean of y 
0.6136364 1.4117647 


[[12]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 2.2711, df = 167.6, p-value = 0.02441
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.6210173 8.8807652
sample estimates:
mean of x mean of y 
 9.530303  4.779412 


[[12]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = 0.71503, df = 224.11, p-value = 0.4753
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -21668532  46348135
sample estimates:
mean of x mean of y 
 46091416  33751615 



[[13]]
[[13]]$event
[1] "Lightning"

[[13]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = 3.0646, df = 255.51, p-value = 0.002413
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.5378441 2.4719256
sample estimates:
mean of x mean of y 
 3.762238  2.257353 


[[13]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 3.7787, df = 237.17, p-value = 0.0001995
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  5.387296 17.123296
sample estimates:
mean of x mean of y 
 23.46853  12.21324 


[[13]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = 2.2863, df = 259.18, p-value = 0.02304
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  235269.6 3156809.0
sample estimates:
mean of x mean of y 
  5173221   3477182 



[[14]]
[[14]]$event
[1] "Rip Current"

[[14]]$deaths

    Welch Two Sample t-test

data:  d_eventEarly$DEATHS_DIRECT and d_eventLate$DEATHS_DIRECT
t = -3.1662, df = 221.47, p-value = 0.001762
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -2.3770092 -0.5531796
sample estimates:
mean of x mean of y 
 3.138211  4.603306 


[[14]]$injuries

    Welch Two Sample t-test

data:  i_eventEarly$INJURIES_DIRECT and i_eventLate$INJURIES_DIRECT
t = 0.23818, df = 194.38, p-value = 0.812
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -1.301225  1.658680
sample estimates:
mean of x mean of y 
 3.186992  3.008264 


[[14]]$prop_damage

    Welch Two Sample t-test

data:  p_eventEarly$ADJ_DAMAGE_PROPERTY and p_eventLate$ADJ_DAMAGE_PROPERTY
t = 1.4881, df = 122.02, p-value = 0.1393
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -619.1328 4368.6884
sample estimates:
  mean of x   mean of y 
1884.679757    9.901952 

t.tests of individual storms comparing frequencies

H0 = storms are not more or less frequent between the two date ranges

HA = storms are more frequent in one of the date ranges than the other

write a function to t-test a list of different storms aggregating by year-month

test_freq <- function(events){
  return_list <- rep(NA,1)
  for (event in events){
    col_name <- paste(event, "_freq", sep="")
    
    early[col_name] <- grepl(event, early$EVENT_TYPE)
    late[col_name] <- grepl(event, late$EVENT_TYPE)
    
    f_early <- aggregate(eval(as.name(col_name)) ~ BEGIN_YEARMONTH, FUN=sum, early)
    f_late <- aggregate(eval(as.name(col_name)) ~ BEGIN_YEARMONTH, FUN=sum, late)
    
    test_list <- list('event' = col_name, 'test' = t.test(f_early[,2], f_late[,2]))
    return_list <- list.append(return_list, test_list)
  }
  return_list <- return_list[!is.na(return_list)]
  return(return_list)
}

using the function to test list of storms except hurricanes

# there are two different ways of putting hurricane into the data so that has to be handled seperately 
test_freq(events[2:length(events)])

$event
[1] "Tornado_freq"

$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = -0.39215, df = 263.07, p-value = 0.6953
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -33.23437  22.19512
sample estimates:
mean of x mean of y 
 113.7847  119.3043 


[[3]]
[[3]]$event
[1] "Heat_freq"

[[3]]$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = -0.61382, df = 227.87, p-value = 0.5399
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -72.73975  38.18480
sample estimates:
mean of x mean of y 
 88.88194 106.15942 



[[4]]
[[4]]$event
[1] "Flash Flood_freq"

[[4]]$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = -1.6803, df = 250.39, p-value = 0.09415
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -108.105874    8.566019
sample estimates:
mean of x mean of y 
 270.5417  320.3116 



[[5]]
[[5]]$event
[1] "Flood_freq"

[[5]]$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = -2.9679, df = 248.24, p-value = 0.003292
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -201.0214  -40.6453
sample estimates:
mean of x mean of y 
 439.1667  560.0000 



[[6]]
[[6]]$event
[1] "Thunderstorm Wind_freq"

[[6]]$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = -2.7053, df = 242.34, p-value = 0.00731
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -727.2114 -114.4003
sample estimates:
mean of x mean of y 
 1046.035  1466.841 



[[7]]
[[7]]$event
[1] "Hail_freq"

[[7]]$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = 0.41126, df = 278.71, p-value = 0.6812
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -199.0246  304.1484
sample estimates:
mean of x mean of y 
1033.0764  980.5145 



[[8]]
[[8]]$event
[1] "High Wind_freq"

[[8]]$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = -1.3293, df = 277.24, p-value = 0.1849
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -85.16946  16.51004
sample estimates:
mean of x mean of y 
 228.0833  262.4130 



[[9]]
[[9]]$event
[1] "Wildfire_freq"

[[9]]$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = -2.9929, df = 268.48, p-value = 0.003021
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -15.125306  -3.121675
sample estimates:
mean of x mean of y 
 20.34028  29.46377 



[[10]]
[[10]]$event
[1] "Lightning_freq"

[[10]]$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = 2.9215, df = 264.73, p-value = 0.003784
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  7.603057 39.034020
sample estimates:
mean of x mean of y 
 69.84028  46.52174 



[[11]]
[[11]]$event
[1] "Rip Current_freq"

[[11]]$test

    Welch Two Sample t-test

data:  f_early[, 2] and f_late[, 2]
t = -3.5412, df = 234.31, p-value = 0.0004804
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -3.1329015 -0.8930647
sample estimates:
mean of x mean of y 
 3.631944  5.644928 

testing hurricanes manually

event <- "Hurricane"

early$hurricane_freq <- grepl(event, early$EVENT_TYPE)
late$hurricane_freq <- grepl(event, late$EVENT_TYPE)

f_early <- aggregate(hurricane_freq ~ BEGIN_YEARMONTH, FUN=sum, early)
f_late <- aggregate(hurricane_freq ~ BEGIN_YEARMONTH, FUN=sum, late)


t.test(f_early$hurricane_freq, f_late$hurricane_freq)

    Welch Two Sample t-test

data:  f_early$hurricane_freq and f_late$hurricane_freq
t = 2.9066, df = 161.13, p-value = 0.004168
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  2.903162 15.208553
sample estimates:
mean of x mean of y 
11.381944  2.326087 

This code is looking at the three types of events we have consistently recorded throughout the 70 years: Hail, Thunderstorm Wind, and Tornadoes. The frequency of these events increase significantly in the later set of years from 1986-2019 but this is most likely due to better or more consistent recording practices by NOAA as opposed to these storms actually being that much more frequent.

yrs1 <- 1950:1985
yrs2 <- 1986:2019

early <- full[full$YEAR %in% yrs1,]
late <- full[full$YEAR %in% yrs2,]

early %>% count(EVENT_TYPE)

l <- late %>% count(EVENT_TYPE)
l[l$EVENT_TYPE %in% c("Hail", "Thunderstorm Wind", "Tornado"),]

---

back to navigation

Time Series Analysis

### animated times series graph over years
year <- as.numeric(substr(full$BEGIN_YEARMONTH, 1, 4))
head(year)

[1] 1950 1950 1950 1950 1950 1950

table(year)         #  data includes records from 1950 - 2019

year
 1950  1951  1952  1953  1954  1955  1956  1957  1958  1959  1960  1961  1962  1963  1964  1965  1966 
  223   269   272   492   609  1413  1703  2184  2213  1813  1945  2246  2389  1968  2348  2835  2388 
 1967  1968  1969  1970  1971  1972  1973  1974  1975  1976  1977  1978  1979  1980  1981  1982  1983 
 2688  3312  2926  3215  3471  2171  4453  5375  4975  3768  3728  3657  4279  6136  4517  7126  8322 
 1984  1985  1986  1987  1988  1989  1990  1991  1992  1993  1994  1995  1996  1997  1998  1999  2000 
 7335  7979  8725  7363  7257 10407 10945 12516 13534  8664 15627 20461 48569 41983 50973 46383 52007 
 2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017 
48875 50936 52956 52409 53976 56400 59010 71190 57398 62804 79091 64503 59985 59465 57788 56003 57024 
 2018  2019 
62351 38586 

max(year) - min(year)  # equals 69, that's good!

[1] 69

full$year <- year
timess <- full %>% count(year)  # matches the values from table function (line 45)
plot(timess, ylab = 'number of records', type = 'l', main = 'Time series for all event types from 1950-2019')

givemedf <- as.data.frame(table(year))
givemedf$year <- as.integer(as.numeric(levels(givemedf$year))[givemedf$year])
#givemedf$year <- as.Date(as.character(givemedf$year), "%Y")

ggplot(givemedf, aes(year, Freq)) +             # animated time series plot for all event types
  geom_line() + 
  geom_point(size = 2) +
  transition_reveal(year) + 
  labs(title = "Year: {frame_along}", x = 'year', y = 'number of records') #+

  [1] "./gganim_plot0001.png" "./gganim_plot0002.png" "./gganim_plot0003.png" "./gganim_plot0004.png"
  [5] "./gganim_plot0005.png" "./gganim_plot0006.png" "./gganim_plot0007.png" "./gganim_plot0008.png"
  [9] "./gganim_plot0009.png" "./gganim_plot0010.png" "./gganim_plot0011.png" "./gganim_plot0012.png"
 [13] "./gganim_plot0013.png" "./gganim_plot0014.png" "./gganim_plot0015.png" "./gganim_plot0016.png"
 [17] "./gganim_plot0017.png" "./gganim_plot0018.png" "./gganim_plot0019.png" "./gganim_plot0020.png"
 [21] "./gganim_plot0021.png" "./gganim_plot0022.png" "./gganim_plot0023.png" "./gganim_plot0024.png"
 [25] "./gganim_plot0025.png" "./gganim_plot0026.png" "./gganim_plot0027.png" "./gganim_plot0028.png"
 [29] "./gganim_plot0029.png" "./gganim_plot0030.png" "./gganim_plot0031.png" "./gganim_plot0032.png"
 [33] "./gganim_plot0033.png" "./gganim_plot0034.png" "./gganim_plot0035.png" "./gganim_plot0036.png"
 [37] "./gganim_plot0037.png" "./gganim_plot0038.png" "./gganim_plot0039.png" "./gganim_plot0040.png"
 [41] "./gganim_plot0041.png" "./gganim_plot0042.png" "./gganim_plot0043.png" "./gganim_plot0044.png"
 [45] "./gganim_plot0045.png" "./gganim_plot0046.png" "./gganim_plot0047.png" "./gganim_plot0048.png"
 [49] "./gganim_plot0049.png" "./gganim_plot0050.png" "./gganim_plot0051.png" "./gganim_plot0052.png"
 [53] "./gganim_plot0053.png" "./gganim_plot0054.png" "./gganim_plot0055.png" "./gganim_plot0056.png"
 [57] "./gganim_plot0057.png" "./gganim_plot0058.png" "./gganim_plot0059.png" "./gganim_plot0060.png"
 [61] "./gganim_plot0061.png" "./gganim_plot0062.png" "./gganim_plot0063.png" "./gganim_plot0064.png"
 [65] "./gganim_plot0065.png" "./gganim_plot0066.png" "./gganim_plot0067.png" "./gganim_plot0068.png"
 [69] "./gganim_plot0069.png" "./gganim_plot0070.png" "./gganim_plot0071.png" "./gganim_plot0072.png"
 [73] "./gganim_plot0073.png" "./gganim_plot0074.png" "./gganim_plot0075.png" "./gganim_plot0076.png"
 [77] "./gganim_plot0077.png" "./gganim_plot0078.png" "./gganim_plot0079.png" "./gganim_plot0080.png"
 [81] "./gganim_plot0081.png" "./gganim_plot0082.png" "./gganim_plot0083.png" "./gganim_plot0084.png"
 [85] "./gganim_plot0085.png" "./gganim_plot0086.png" "./gganim_plot0087.png" "./gganim_plot0088.png"
 [89] "./gganim_plot0089.png" "./gganim_plot0090.png" "./gganim_plot0091.png" "./gganim_plot0092.png"
 [93] "./gganim_plot0093.png" "./gganim_plot0094.png" "./gganim_plot0095.png" "./gganim_plot0096.png"
 [97] "./gganim_plot0097.png" "./gganim_plot0098.png" "./gganim_plot0099.png" "./gganim_plot0100.png"
attr(,"frame_vars")
    frame nframes progress frame_along
1       1     100     0.01        1950
2       2     100     0.02        1951
3       3     100     0.03        1951
4       4     100     0.04        1952
5       5     100     0.05        1953
6       6     100     0.06        1953
7       7     100     0.07        1954
8       8     100     0.08        1955
9       9     100     0.09        1956
10     10     100     0.10        1956
11     11     100     0.11        1957
12     12     100     0.12        1958
13     13     100     0.13        1958
14     14     100     0.14        1959
15     15     100     0.15        1960
16     16     100     0.16        1960
17     17     100     0.17        1961
18     18     100     0.18        1962
19     19     100     0.19        1963
20     20     100     0.20        1963
21     21     100     0.21        1964
22     22     100     0.22        1965
23     23     100     0.23        1965
24     24     100     0.24        1966
25     25     100     0.25        1967
26     26     100     0.26        1967
27     27     100     0.27        1968
28     28     100     0.28        1969
29     29     100     0.29        1970
30     30     100     0.30        1970
31     31     100     0.31        1971
32     32     100     0.32        1972
33     33     100     0.33        1972
34     34     100     0.34        1973
35     35     100     0.35        1974
36     36     100     0.36        1974
37     37     100     0.37        1975
38     38     100     0.38        1976
39     39     100     0.39        1976
40     40     100     0.40        1977
41     41     100     0.41        1978
42     42     100     0.42        1979
43     43     100     0.43        1979
44     44     100     0.44        1980
45     45     100     0.45        1981
46     46     100     0.46        1981
47     47     100     0.47        1982
48     48     100     0.48        1983
49     49     100     0.49        1983
50     50     100     0.50        1984
51     51     100     0.51        1985
52     52     100     0.52        1986
53     53     100     0.53        1986
54     54     100     0.54        1987
55     55     100     0.55        1988
56     56     100     0.56        1988
57     57     100     0.57        1989
58     58     100     0.58        1990
59     59     100     0.59        1990
60     60     100     0.60        1991
61     61     100     0.61        1992
62     62     100     0.62        1993
63     63     100     0.63        1993
64     64     100     0.64        1994
65     65     100     0.65        1995
66     66     100     0.66        1995
67     67     100     0.67        1996
68     68     100     0.68        1997
69     69     100     0.69        1997
70     70     100     0.70        1998
71     71     100     0.71        1999
72     72     100     0.72        1999
73     73     100     0.73        2000
74     74     100     0.74        2001
75     75     100     0.75        2002
76     76     100     0.76        2002
77     77     100     0.77        2003
78     78     100     0.78        2004
79     79     100     0.79        2004
80     80     100     0.80        2005
81     81     100     0.81        2006
82     82     100     0.82        2006
83     83     100     0.83        2007
84     84     100     0.84        2008
85     85     100     0.85        2009
86     86     100     0.86        2009
87     87     100     0.87        2010
88     88     100     0.88        2011
89     89     100     0.89        2011
90     90     100     0.90        2012
91     91     100     0.91        2013
92     92     100     0.92        2013
93     93     100     0.93        2014
94     94     100     0.94        2015
95     95     100     0.95        2016
96     96     100     0.96        2016
97     97     100     0.97        2017
98     98     100     0.98        2018
99     99     100     0.99        2018
100   100     100     1.00        2019
                                                                                     frame_source
1   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0001.png
2   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0002.png
3   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0003.png
4   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0004.png
5   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0005.png
6   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0006.png
7   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0007.png
8   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0008.png
9   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0009.png
10  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0010.png
11  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0011.png
12  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0012.png
13  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0013.png
14  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0014.png
15  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0015.png
16  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0016.png
17  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0017.png
18  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0018.png
19  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0019.png
20  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0020.png
21  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0021.png
22  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0022.png
23  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0023.png
24  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0024.png
25  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0025.png
26  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0026.png
27  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0027.png
28  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0028.png
29  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0029.png
30  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0030.png
31  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0031.png
32  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0032.png
33  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0033.png
34  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0034.png
35  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0035.png
36  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0036.png
37  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0037.png
38  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0038.png
39  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0039.png
40  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0040.png
41  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0041.png
42  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0042.png
43  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0043.png
44  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0044.png
45  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0045.png
46  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0046.png
47  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0047.png
48  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0048.png
49  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0049.png
50  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0050.png
51  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0051.png
52  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0052.png
53  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0053.png
54  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0054.png
55  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0055.png
56  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0056.png
57  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0057.png
58  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0058.png
59  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0059.png
60  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0060.png
61  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0061.png
62  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0062.png
63  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0063.png
64  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0064.png
65  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0065.png
66  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0066.png
67  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0067.png
68  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0068.png
69  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0069.png
70  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0070.png
71  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0071.png
72  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0072.png
73  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0073.png
74  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0074.png
75  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0075.png
76  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0076.png
77  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0077.png
78  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0078.png
79  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0079.png
80  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0080.png
81  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0081.png
82  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0082.png
83  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0083.png
84  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0084.png
85  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0085.png
86  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0086.png
87  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0087.png
88  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0088.png
89  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0089.png
90  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0090.png
91  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0091.png
92  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0092.png
93  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0093.png
94  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0094.png
95  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0095.png
96  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0096.png
97  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0097.png
98  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0098.png
99  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0099.png
100 /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da773ef3a526/gganim_plot0100.png

  #anim_save('all_event_new.gif')             # Comment this line if you don't want to save it as a gif locally

fancy <- as.data.frame(head(sort(table(full$EVENT_TYPE)[table(full$EVENT_TYPE) > 10000],decreasing = T),9))
colnames(fancy)[1] <- "event_type"
fancy                               # Top 9 event types with most records

flag <- fancy[,1]
fullflag <- full[full$EVENT_TYPE==flag,]

longer object length is not a multiple of shorter object lengthlonger object length is not a multiple of shorter object length

table(fullflag$EVENT_TYPE,fullflag$year)      # 6 event types with 0 records until 1995 (Drought, Flash Flood, HS, HW, WS, WW)

                   
                    1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966
  Drought              0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Flash Flood          0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Hail                 0    0    0    0    0   39   47   56   74   56   62   87  103   70   82   88   88
  Heavy Snow           0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  High Wind            0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Thunderstorm Wind    0    0    0    0    0   42   89   74   79   71   78   86   88   88   93  108  111
  Tornado             25   30   30   55   67   75   67  102   61   57   71   87   71   64   80  113   63
  Winter Storm         0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Winter Weather       0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
                   
                    1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983
  Drought              0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Flash Flood          0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Hail                83  113   85   79   99   73  110  184  154  128  110  114  151  213  164  285  260
  Heavy Snow           0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  High Wind            0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Thunderstorm Wind   95  175  173  194  167   77  224  293  305  184  189  210  241  344  241  400  564
  Tornado            106   81   77   76  116   87  123  128  103  109   98   99  108  110   93  120  103
  Winter Storm         0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Winter Weather       0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
                   
                    1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000
  Drought              0    0    0    0    0    0    0    0    0    0    0    0   72   45  192  285  254
  Flash Flood          0    0    0    0    0    0    0    0    0    0    0    0  372  314  401  295  302
  Hail               316  392  403  289  287  428  409  509  651  462  755  932 1219  952 1402 1127 1233
  Heavy Snow           0    0    0    0    0    0    0    0    0    0    0    0  480  346  258  279  444
  High Wind            0    0    0    0    0    0    0    0    0    0    0    0  415  301  248  354  332
  Thunderstorm Wind  388  443  444  466  455  637  693  733  741  447  853 1226 1137 1094 1520 1118 1349
  Tornado            110   98   85   86   92  102  147  145  157   74   90  153  145  123  141  189  131
  Winter Storm         0    0    0    0    0    0    0    0    0    0    0    0  351  357  218  336  458
  Winter Weather       0    0    0    0    0    0    0    0    0    0    0    0   72  124   89  130   51
                   
                    2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
  Drought            199  218  111   60  170  290  398  216  158  175  614  878  584  301  149  267  136
  Flash Flood        327  360  454  533  329  297  412  430  444  447  396  257  530  321  566  417  386
  Hail              1286 1373 1542 1475 1498 1844 1401 1965 1483 1241 1938 1496 1045 1120 1025 1034 1148
  Heavy Snow         315  371  318  240  305  170  285  309  256  397  305  159  252  305  292  156  193
  High Wind          320  354  244  268  264  308  251  413  323  311  354  379  247  346  230  350  402
  Thunderstorm Wind 1311 1321 1378 1332 1307 1490 1468 1861 1445 1749 2324 1795 1660 1546 1588 1746 1830
  Tornado            133  137  168  213  153  134  135  212  166  157  242  119  117  111  124  104  193
  Winter Storm       278  373  402  351  334  215  346  457  373  424  352  196  415  437  402  259  241
  Winter Weather      83   42  192  182  222  171  376  447  312  478  417  308  531  534  460  304  285
                   
                    2018 2019
  Drought            273   17
  Flash Flood        474  229
  Hail               901  602
  Heavy Snow         239  218
  High Wind          328  256
  Thunderstorm Wind 1669 1102
  Tornado            131  148
  Winter Storm       397  241
  Winter Weather     491  269

givemedf2 <- as.data.frame(table(fullflag$EVENT_TYPE,fullflag$year))
givemedf2$Var2 <- as.integer(as.numeric(levels(givemedf2$Var2))[givemedf2$Var2])
#givemedf2$Var2 <- as.Date(as.character(givemedf2$Var2), "%Y")

ggplot(givemedf2, aes(Var2, Freq, group=Var1)) +              # animated time series plot for top 9 event types with most records
  geom_line(aes(col = Var1), size = 1) +
  scale_colour_discrete("event type") +
  geom_point(size = 2) +
  transition_reveal(Var2) + 
  labs(title = "Year: {frame_along}", x = 'year', y = 'number of records') #+

  [1] "./gganim_plot0001.png" "./gganim_plot0002.png" "./gganim_plot0003.png" "./gganim_plot0004.png"
  [5] "./gganim_plot0005.png" "./gganim_plot0006.png" "./gganim_plot0007.png" "./gganim_plot0008.png"
  [9] "./gganim_plot0009.png" "./gganim_plot0010.png" "./gganim_plot0011.png" "./gganim_plot0012.png"
 [13] "./gganim_plot0013.png" "./gganim_plot0014.png" "./gganim_plot0015.png" "./gganim_plot0016.png"
 [17] "./gganim_plot0017.png" "./gganim_plot0018.png" "./gganim_plot0019.png" "./gganim_plot0020.png"
 [21] "./gganim_plot0021.png" "./gganim_plot0022.png" "./gganim_plot0023.png" "./gganim_plot0024.png"
 [25] "./gganim_plot0025.png" "./gganim_plot0026.png" "./gganim_plot0027.png" "./gganim_plot0028.png"
 [29] "./gganim_plot0029.png" "./gganim_plot0030.png" "./gganim_plot0031.png" "./gganim_plot0032.png"
 [33] "./gganim_plot0033.png" "./gganim_plot0034.png" "./gganim_plot0035.png" "./gganim_plot0036.png"
 [37] "./gganim_plot0037.png" "./gganim_plot0038.png" "./gganim_plot0039.png" "./gganim_plot0040.png"
 [41] "./gganim_plot0041.png" "./gganim_plot0042.png" "./gganim_plot0043.png" "./gganim_plot0044.png"
 [45] "./gganim_plot0045.png" "./gganim_plot0046.png" "./gganim_plot0047.png" "./gganim_plot0048.png"
 [49] "./gganim_plot0049.png" "./gganim_plot0050.png" "./gganim_plot0051.png" "./gganim_plot0052.png"
 [53] "./gganim_plot0053.png" "./gganim_plot0054.png" "./gganim_plot0055.png" "./gganim_plot0056.png"
 [57] "./gganim_plot0057.png" "./gganim_plot0058.png" "./gganim_plot0059.png" "./gganim_plot0060.png"
 [61] "./gganim_plot0061.png" "./gganim_plot0062.png" "./gganim_plot0063.png" "./gganim_plot0064.png"
 [65] "./gganim_plot0065.png" "./gganim_plot0066.png" "./gganim_plot0067.png" "./gganim_plot0068.png"
 [69] "./gganim_plot0069.png" "./gganim_plot0070.png" "./gganim_plot0071.png" "./gganim_plot0072.png"
 [73] "./gganim_plot0073.png" "./gganim_plot0074.png" "./gganim_plot0075.png" "./gganim_plot0076.png"
 [77] "./gganim_plot0077.png" "./gganim_plot0078.png" "./gganim_plot0079.png" "./gganim_plot0080.png"
 [81] "./gganim_plot0081.png" "./gganim_plot0082.png" "./gganim_plot0083.png" "./gganim_plot0084.png"
 [85] "./gganim_plot0085.png" "./gganim_plot0086.png" "./gganim_plot0087.png" "./gganim_plot0088.png"
 [89] "./gganim_plot0089.png" "./gganim_plot0090.png" "./gganim_plot0091.png" "./gganim_plot0092.png"
 [93] "./gganim_plot0093.png" "./gganim_plot0094.png" "./gganim_plot0095.png" "./gganim_plot0096.png"
 [97] "./gganim_plot0097.png" "./gganim_plot0098.png" "./gganim_plot0099.png" "./gganim_plot0100.png"
attr(,"frame_vars")
    frame nframes progress frame_along
1       1     100     0.01        1950
2       2     100     0.02        1951
3       3     100     0.03        1951
4       4     100     0.04        1952
5       5     100     0.05        1953
6       6     100     0.06        1953
7       7     100     0.07        1954
8       8     100     0.08        1955
9       9     100     0.09        1956
10     10     100     0.10        1956
11     11     100     0.11        1957
12     12     100     0.12        1958
13     13     100     0.13        1958
14     14     100     0.14        1959
15     15     100     0.15        1960
16     16     100     0.16        1960
17     17     100     0.17        1961
18     18     100     0.18        1962
19     19     100     0.19        1963
20     20     100     0.20        1963
21     21     100     0.21        1964
22     22     100     0.22        1965
23     23     100     0.23        1965
24     24     100     0.24        1966
25     25     100     0.25        1967
26     26     100     0.26        1967
27     27     100     0.27        1968
28     28     100     0.28        1969
29     29     100     0.29        1970
30     30     100     0.30        1970
31     31     100     0.31        1971
32     32     100     0.32        1972
33     33     100     0.33        1972
34     34     100     0.34        1973
35     35     100     0.35        1974
36     36     100     0.36        1974
37     37     100     0.37        1975
38     38     100     0.38        1976
39     39     100     0.39        1976
40     40     100     0.40        1977
41     41     100     0.41        1978
42     42     100     0.42        1979
43     43     100     0.43        1979
44     44     100     0.44        1980
45     45     100     0.45        1981
46     46     100     0.46        1981
47     47     100     0.47        1982
48     48     100     0.48        1983
49     49     100     0.49        1983
50     50     100     0.50        1984
51     51     100     0.51        1985
52     52     100     0.52        1986
53     53     100     0.53        1986
54     54     100     0.54        1987
55     55     100     0.55        1988
56     56     100     0.56        1988
57     57     100     0.57        1989
58     58     100     0.58        1990
59     59     100     0.59        1990
60     60     100     0.60        1991
61     61     100     0.61        1992
62     62     100     0.62        1993
63     63     100     0.63        1993
64     64     100     0.64        1994
65     65     100     0.65        1995
66     66     100     0.66        1995
67     67     100     0.67        1996
68     68     100     0.68        1997
69     69     100     0.69        1997
70     70     100     0.70        1998
71     71     100     0.71        1999
72     72     100     0.72        1999
73     73     100     0.73        2000
74     74     100     0.74        2001
75     75     100     0.75        2002
76     76     100     0.76        2002
77     77     100     0.77        2003
78     78     100     0.78        2004
79     79     100     0.79        2004
80     80     100     0.80        2005
81     81     100     0.81        2006
82     82     100     0.82        2006
83     83     100     0.83        2007
84     84     100     0.84        2008
85     85     100     0.85        2009
86     86     100     0.86        2009
87     87     100     0.87        2010
88     88     100     0.88        2011
89     89     100     0.89        2011
90     90     100     0.90        2012
91     91     100     0.91        2013
92     92     100     0.92        2013
93     93     100     0.93        2014
94     94     100     0.94        2015
95     95     100     0.95        2016
96     96     100     0.96        2016
97     97     100     0.97        2017
98     98     100     0.98        2018
99     99     100     0.99        2018
100   100     100     1.00        2019
                                                                                    frame_source
1   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0001.png
2   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0002.png
3   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0003.png
4   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0004.png
5   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0005.png
6   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0006.png
7   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0007.png
8   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0008.png
9   /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0009.png
10  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0010.png
11  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0011.png
12  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0012.png
13  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0013.png
14  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0014.png
15  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0015.png
16  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0016.png
17  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0017.png
18  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0018.png
19  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0019.png
20  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0020.png
21  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0021.png
22  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0022.png
23  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0023.png
24  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0024.png
25  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0025.png
26  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0026.png
27  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0027.png
28  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0028.png
29  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0029.png
30  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0030.png
31  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0031.png
32  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0032.png
33  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0033.png
34  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0034.png
35  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0035.png
36  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0036.png
37  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0037.png
38  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0038.png
39  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0039.png
40  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0040.png
41  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0041.png
42  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0042.png
43  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0043.png
44  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0044.png
45  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0045.png
46  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0046.png
47  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0047.png
48  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0048.png
49  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0049.png
50  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0050.png
51  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0051.png
52  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0052.png
53  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0053.png
54  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0054.png
55  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0055.png
56  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0056.png
57  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0057.png
58  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0058.png
59  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0059.png
60  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0060.png
61  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0061.png
62  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0062.png
63  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0063.png
64  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0064.png
65  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0065.png
66  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0066.png
67  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0067.png
68  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0068.png
69  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0069.png
70  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0070.png
71  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0071.png
72  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0072.png
73  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0073.png
74  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0074.png
75  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0075.png
76  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0076.png
77  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0077.png
78  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0078.png
79  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0079.png
80  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0080.png
81  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0081.png
82  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0082.png
83  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0083.png
84  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0084.png
85  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0085.png
86  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0086.png
87  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0087.png
88  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0088.png
89  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0089.png
90  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0090.png
91  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0091.png
92  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0092.png
93  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0093.png
94  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0094.png
95  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0095.png
96  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0096.png
97  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0097.png
98  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0098.png
99  /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0099.png
100 /var/folders/n2/lc639sxd11v3ckv2rbnwxt8r0000gn/T//RtmpQsQnnu/da77d37966b/gganim_plot0100.png

  #anim_save('top9_new.gif')       # Comment this line if you don't want to save it as a gif locally

### severe weather in Colorado predictive model
full2 <- full[(full$STATE=='COLORADO') & (full$year != 2019),]     # subset all Colorado records & get rid of 2019 (since no year-long data)
fancy <- as.data.frame(head(sort(table(full2$EVENT_TYPE),decreasing = T),9))     # Top 9 most happened severe weather in Colorado
colnames(fancy)[1] <- "event_type"                                    # Hail, Winter Storm, Winter Weather, Thunderstorm...
fancy

timess2 <- full2 %>% count(year)
plot(timess2, ylab = 'number of records', type = 'l', main = 'Time series for all event types happened in Colorado from 1950-2018')     # Time series plot for all severe weather types in Colorado

par(mfrow = c(3,3))

arnumber <- rep(NA, times=nrow(fancy))
pred19 <- rep(NA, times=nrow(fancy))
pred20 <- rep(NA, times=nrow(fancy))

for (i in 1:nrow(fancy)){
  fullsub <- full2[full2$EVENT_TYPE==fancy[i,1],]
  newtimess <- fullsub %>% count(year)
  newtimess <- as.data.frame(newtimess)
  plot(newtimess, ylab = 'number of records', type = 'l', main = paste(fancy[i,1]))
  ar1 <- ar(newtimess[,2])
  arnumber[i] <- ar1$order
  predRe <- predict(ar1,n.ahead = 2)
  pred19[i] <- predRe$pred[1]
  pred20[i] <- predRe$pred[2]
}



#################### Example of Hail (Colorado)
par(mfrow = c(1,1))

fullsub <- full2[full2$EVENT_TYPE==fancy[1,1],]
newtimess <- fullsub %>% count(year)
newtimess <- as.data.frame(newtimess)
plot(newtimess, ylab = 'number of records', type = 'l', main = paste(fancy[1,1]))

par(mfrow = c(1,2))

acf(newtimess[,2])
pacf(newtimess[,2])

## Since pacf displays a sharp cutoff and acf displays a slow decay, this series displays an "ar signature"
## by reading both pacf and acf, we are confident to use ar(15) model since PACF cut off is at lag 15.


#################### Example of Drought (Colorado)
par(mfrow = c(1,1))

fullsub <- full2[full2$EVENT_TYPE==fancy[6,1],]
newtimess <- fullsub %>% count(year)
newtimess <- as.data.frame(newtimess)
plot(newtimess, ylab = 'number of records', type = 'l', main = paste(fancy[6,1]))

par(mfrow = c(1,2))

acf(newtimess[,2])
pacf(newtimess[,2])

## We can see that for Drought, there is no strong geometry decay pattern for both acf and pacf, so it's not a good idea
## to use ar function on Drought (similar situation for High Wind, Flash Flood). It's also interesting for ar model of 
## this type of weather, the automatically selected order window is 0. And the 2019 prediction ^ 2020 prediction are the same.

arnumber_new <- arnumber
pred19_new <- floor(pred19)
pred20_new <- floor(pred20)
fancytype <- as.character(fancy[,1])

givemetable <- cbind(fancytype, arnumber_new, pred19_new, pred20_new)
colnames(givemetable)[1] <- "Event Type"
colnames(givemetable)[2] <- "Lag Window"
colnames(givemetable)[3] <- "2019 Prediction"
colnames(givemetable)[4] <- "2020 Prediction"
xtable(givemetable)

% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Sat Nov 23 17:29:40 2019
\begin{table}[ht]
\centering
\begin{tabular}{rllll}
  \hline
 & Event Type & Lag Window & 2019 Prediction & 2020 Prediction \\ 
  \hline
1 & Hail & 2 & 468 & 485 \\ 
  2 & Winter Storm & 1 & 148 & 179 \\ 
  3 & Winter Weather & 1 & 274 & 262 \\ 
  4 & Thunderstorm Wind & 6 & 76 & 87 \\ 
  5 & High Wind & 0 & 110 & 110 \\ 
  6 & Drought & 0 & 145 & 145 \\ 
  7 & Tornado & 8 & 40 & 41 \\ 
  8 & Heavy Snow & 1 & 44 & 60 \\ 
  9 & Flash Flood & 0 & 51 & 51 \\ 
   \hline
\end{tabular}
\end{table}

---

back to navigation

Predictive Modelling

### severe weather in Colorado predictive model
full2 <- full[(full$STATE=='COLORADO') & (full$year != 2019),]     # subset all Colorado records & get rid of 2019 (since no year-long data)
fancy <- as.data.frame(head(sort(table(full2$EVENT_TYPE),decreasing = T),9))     # Top 9 most happened severe weather in Colorado
colnames(fancy)[1] <- "event_type"                                    # Hail, Winter Storm, Winter Weather, Thunderstorm...
fancy

timess2 <- full2 %>% count(year)
plot(timess2, ylab = 'number of records', type = 'l', main = 'Time series for all event types happened in Colorado from 1950-2018')     # Time series plot for all severe weather types in Colorado

par(mfrow = c(3,3))

arnumber <- rep(NA, times=nrow(fancy))
pred19 <- rep(NA, times=nrow(fancy))
pred20 <- rep(NA, times=nrow(fancy))

for (i in 1:nrow(fancy)){
  fullsub <- full2[full2$EVENT_TYPE==fancy[i,1],]
  newtimess <- fullsub %>% count(year)
  newtimess <- as.data.frame(newtimess)
  plot(newtimess, ylab = 'number of records', type = 'l', main = paste(fancy[i,1]))
  ar1 <- ar(newtimess[,2])
  arnumber[i] <- ar1$order
  predRe <- predict(ar1,n.ahead = 2)
  pred19[i] <- predRe$pred[1]
  pred20[i] <- predRe$pred[2]
}

#################### Example of Hail (Colorado)
par(mfrow = c(1,1))
fullsub <- full2[full2$EVENT_TYPE==fancy[1,1],]
newtimess <- fullsub %>% count(year)
newtimess <- as.data.frame(newtimess)
plot(newtimess, ylab = 'number of records', type = 'l', main = paste(fancy[1,1]))

par(mfrow = c(1,2))

acf(newtimess[,2])
pacf(newtimess[,2])

## Since pacf displays a sharp cutoff and acf displays a slow decay, this series displays an "ar signature"
## by reading both pacf and acf, we are confident to use ar(15) model since PACF cut off is at lag 15.

#################### Example of Drought (Colorado)
par(mfrow = c(1,1))
fullsub <- full2[full2$EVENT_TYPE==fancy[6,1],]
newtimess <- fullsub %>% count(year)
newtimess <- as.data.frame(newtimess)
plot(newtimess, ylab = 'number of records', type = 'l', main = paste(fancy[6,1]))

par(mfrow = c(1,2))

acf(newtimess[,2])
pacf(newtimess[,2])

## We can see that for Drought, there is no strong geometry decay pattern for both acf and pacf, so it's not a good idea
## to use ar function on Drought (similar situation for High Wind, Flash Flood). It's also interesting for ar model of 
## this type of weather, the automatically selected order window is 0. And the 2019 prediction ^ 2020 prediction are the same.

arnumber_new <- arnumber
pred19_new <- floor(pred19)
pred20_new <- floor(pred20)
fancytype <- as.character(fancy[,1])

givemetable <- cbind(fancytype, arnumber_new, pred19_new, pred20_new)
colnames(givemetable)[1] <- "Event Type"
colnames(givemetable)[2] <- "Lag Window"
colnames(givemetable)[3] <- "2019 Prediction"
colnames(givemetable)[4] <- "2020 Prediction"
xtable(givemetable)

% latex table generated in R 3.6.1 by xtable 1.8-4 package
% Sat Nov 23 17:33:42 2019
\begin{table}[ht]
\centering
\begin{tabular}{rllll}
  \hline
 & Event Type & Lag Window & 2019 Prediction & 2020 Prediction \\ 
  \hline
1 & Hail & 2 & 468 & 485 \\ 
  2 & Winter Storm & 1 & 148 & 179 \\ 
  3 & Winter Weather & 1 & 274 & 262 \\ 
  4 & Thunderstorm Wind & 6 & 76 & 87 \\ 
  5 & High Wind & 0 & 110 & 110 \\ 
  6 & Drought & 0 & 145 & 145 \\ 
  7 & Tornado & 8 & 40 & 41 \\ 
  8 & Heavy Snow & 1 & 44 & 60 \\ 
  9 & Flash Flood & 0 & 51 & 51 \\ 
   \hline
\end{tabular}
\end{table}

---

back to navigation

Creating the fullWeather.csv file from the downloaded NOAA files

Rename files in details folder to allow for binding of data (only needed to be performed once)

Bind files from every year together into dataframe labelled “full” (70 years worth of data!) then export to csv file for ease of use and sharing purposes (only needed to be performed once)